STATE-DEPENDENT FOSTER-LYAPUNOV CRITERIA FOR 
SUBGEOMETRIC CONVERGENCE OF MARKOV CHAINS 



S. CONNOR AND G. FORT 

Abstract. We consider a form of state-dependent drift condition for a general Markov 
chain, whereby the chain subsampled at some deterministic time satisfies a geometric 
Foster-Lyapunov condition. We present sufficient criteria for such a drift condition to 
exist, and use these to partially answer a question posed in Q] concerning the existence 
CD . of so-called 'tame' Markov chains. Furthermore, we show that our 'subsampled drift 



(N 



condition' implies the existence of finite moments for the return time to a small set. 
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Cn| ■ 1. Introduction and notation 

> 

ly-j | Let n > 0} be a time-homogeneous Markov chain on a state space X, with transition 



kernel P. Our goal in this paper is to develop a new criterion for determining the ergodic 
properties of <!?. Specifically, we consider a form of state- dependent drift condition for 
■ <£, whereby the chain subsampled at some deterministic time satisfies a geometric Foster- 
Lyapunov condition. Drift conditions are classical tools to prove the stability of Markov 
chains. Most of the literature addresses the case when the drift inequality is satisfied 
^ ■ by the kernel P [15I lol . 0, Q]. Nevertheless, depending upon the application, it may 
be far easier to prove a drift condition for a state-dependent iterated kernel than for P 
itself (see e.g. and section EOj) . State-dependent drift conditions for phi- irreducible 
chains were originally studied by Meyn and Tweedie [lj], who gave criteria for $ to be 
Harris-recurrent, positive Harris-recurrent and geometrically ergodic. The drift condition 
that we develop in this paper is different from those in [LJ], as will be highlighted in 
Section T3.21 and can be used to infer a greater range of convergence rates for $ (including 
both geometric and subgeometric rates). When applied to subgeometric rates, the present 
work extends the theory about subgeometric chains: in [21I] (resp. 0]), it is discussed 
how a nested family of drift conditions (resp. a single drift) on the kernel P is related 
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to the control of modulated moments of the return-time to some set. In this paper, we 
provide similar drift criteria in terms of state-dependent iterates of the transition kernel. 
Control of such return-times is a key step to establish ergodicity and, more generally, limit 
theorems for the chains. In this paper, we also address a converse result and show how a 
state-dependent drift condition can be deduced from the convergence of the iterates of P. 



Such a converse result exists for geometric chains (see [15() but, to our best knowledge, 
this is pioneering work for subgeometric chains. 



We begin with a little notation; the unfamiliar reader can refer to 15[. For any non- 
negative function / and n G N we write P n f(x) for J P n (x,dy)f(y) where P n (x,dy) 
denotes the n-step transition probability kernel; and for a signed measure [i we write 
ju(/) = J fj,(dy) f (y) . The norm is defined as SU P{ 5 :| 9 |</} IMff)!- This generalises the 
total variation norm, || • ||tv = II ' 111- The first return time to a set A is denoted by T4 := 
inf{n > 1, $ n G .A} and the hitting-time on A is denoted by o~a := inf{n > 0, <& n G A}. A 
set C is called small (or i/-small) if there exist some non-trivial measure v and constant 
e > such that P(x,-) > £v(-) for all x G C. Any u-finite measure ir satisfying ir = irP 
is called invariant. An aperiodic chain $ that possesses a finite invariant measure tt is 
ergodic and for any x, ||P n (x, •) — 7t||tv ^ as n — > 00 ( 15l. Theorem 13.0.1]). This 
condition is also equivalent to the existence of a moment for the return time to some 
accessible small set C: sup xgC E x [tq] < 00. <I> is said to be geometrically ergodic if there 
exist a function V : X — > [0, 00) and constants 7 G (0, 1), R < 00, such that for any x G X, 
||P n (x,-) — vr(-)||y < R V(x)"f n . This is equivalent to the existence of a scale function 
V : X — > [1, 00), a small set C, and constants (3 G (0, 1), b < 00, such that 

PV{x) < (5V{x) +bl c (x) , (1) 

where 1,4 is the indicator function of the set A. This geometric drift condition is also 
equivalent to the existence of an exponential moment for the return time to the set C: 
su Px£C^'x[(3~ TC ] < 00 (0> Chapter 15]). More generally, <I> is said to be (f,r)-ergodic if 
there exist functions r : N — »■ [1, 00) and / : X — > [1, 00) such that, for all x in a full and 
absorbing set, r(n)\\P n (x, •) — vr ( - ) 1 1 y — > as n — > 00. 

In this paper we will be studying subgeometrically ergodic chains; the class A of sub- 



geometric rates r = {r(n),n > 0} is defined in [19( as follows: call Ao the set of rate 
functions ro = {ro(n),n > 0} such that ro(0) > 1, n 1— > ro(n) is nondecreasing and 
lim n ^ 00 In ro (n) jn J, 0. Then r G A iff r is nonnegative, non-decreasing and there exists 
ro G Ao such that liuin-^oo r(n) /ro(n) = 1. Sufficient drift conditions for (/, r)-ergodicity, 
relative to the one-step transition kernel P exist in the literature 

HQ. 

The converse 



result is, to our best knowledge, an open question (except when / = 1, see |2ll]). 



The remainder of this paper is laid out as follows. In Section [2] we consider when it is 
possible to find functions n : X — > N* and V : X — ► [1, oo), and E (0, 1) such that 

P n ^V{x) < 0V{x) + bl e (x) . (2) 

That is, such that the chain $ subsampled at time n(x) exhibits a geometric drift condition. 
The sufficient conditions presented are ultimately based on the existence of moments for 
the return time to a small set. In Section [3] we consider the inverse problem: starting 
from a drift condition of the form ([2]), what can be said about the existence of moments 
of re? In Section H] these results are applied to the classification of tame chains, and to 
a discrete-time process forming part of a perfect simulation algorithm for such chains. In 
Section O it is shown how these results can be used to prove the subgeometric ergodicity 
of strong Markov processes. Proofs of the main results are provided in Section [6j 

2. Foster-Lyapunov drift inequalities under subsampling 

We first consider when it is possible to deterministically subsample a chain $ at rate 
n, in order to produce a Foster-Lyapunov drift condition, i.e. an inequality of the form 
P n ( x )V(x) < (3V(x)+blc{x) for some function V : X — ► [1, oo), constants G (0, 1), b < oo 
and a measurable set C. The main result of this section is the following generalisation of 
[l|, Theorem 5.26]. It shows how a Foster-Lyapunov drift condition may be established 
for a subsampled chain from knowledge of the rate of convergence of the signed measures 
{P n (x,-) - P n (x',-),n > 0}. 

Theorem 2.1. Assume that there exists a non- decreasing function r : N — > (0, oo) with 
lim^oo r(k) = oo, some measurable functions W, V : X — > [1, oo) and a constant C < oo 
such that 

V(i,i')eXxX, r(k) \\P k (x,-)- P k (x',-)\\ w <C(V(x) + V{x')) , (3) 

3x SX, sup P k W{x ) < oo . (4) 

fc>0 

Let (3 G (0, 1) and n : X -> N satisfy n{x) > r" 1 where r" 1 ^) := inf{x G 

N, r(x) > t} denotes the generalized inverse of r. Then there exists b < oo such that 
P n ^W(x) < pW(x) + b. In addition, for any < 0' < 1 with C := {x G X,W(x) < 
b(0>-0)^}, 

p n ( x )W < 0'W + btc- (5) 
Proof. From (|3|) and ([!]), we have for any x G X, k € N, 

P k W(x) < ^V(x) + P k W(x ) + ^V(x ) < ^V(x) + b , 
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where b : = sup fc>0 P k W(xo) + C V ^^ . By definition of x i— ► n(x), CV(x)/r(n(x)) < 
l3W(x). This yields P n ^W <f]W + b = f3'W+((3 - 0') W + b, and Q3 - ft) W + b < on 
C c . Since lim^^ r(k) = oo, the set {x G N, r(x) > t} is non-empty whatever t > 0. □ 

2.1. Uniformly ergodic chains. When assumption © holds for some bounded func- 
tion V, we have lim n sup( Xi2 ./) gXx x \\P n ( x > ') ~~ P n {x', -)||tv = 0. Then classical results 
on the Dobrushin coefficient imply that P admits a unique invariant distribution ir and 
sup^gx ||-P n (x, •) — vt||tv < P n for some p G (0,1). Hence the drift condition ([1]) holds 



for some bounded function V ( 15|, Theorem 16.0.1]), and this implies ([5]) with n(x) = c 
for any c G N. We are able to retrieve this result too from our result. Applying The- 
orem 12.11 with W = u! and n(x) = n* such that n* > r _1 (C (5~ l supx V) we have 
P n *W < + 6. By classical computations (see e.g. the proof of [15I . Theorem 16.1.4]), 
this yields PW < W + b(3 l / n *- l n~ l with u < W < lo(3~ 1 ; hence © holds with 

n(x) = c for some (and thus any) constant c. 

In the sequel, we do not impose boundedness on V, thus allowing chains which are not 
necessarily uniformly ergodic. 

2.2. Sufficient conditions for Assumptions ©-(jU). When P is phi-irreducible and 
aperiodic, assumption is implied by any one of the following equivalent conditions (see 



< 00; 



m, Theorem 14.0.1]): 

(i) P possesses a unique invariant probability ir and ir(W) < 00; 

(ii) there exists a small set C such that sup^^E^ Y^k=o W(^fc) 

(iii) there exist a function U : X — ► (0, 00] finite at some x+ G X, a constant 6 < 00 and a 
small set C such that PU < U - W + bt c . 

The main difficulty is to prove ([3]); Proposition 12.21 provides sufficient conditions. 

Proposition 2.2. Let P be a phi-irreducible and aperiodic transition kernel. Assume that 
there exist a small set C, measurable functions W, V : X — > [1, 00) and a constant b < 00 
suc/t t/iai sup c V < 00, and 

f PF(x) < y(x) - W(x) + bt c (x) , 
\ PT^(x) < W(x) + bl c (x) . 

Then (i) there exists x$ G X such that sup,j>o P n W(xo) < 00; and (ii) there exists a 
constant C < 00 such that for any (x,x ! ) G X 2 , 

n \\P n (x, •) - P n (x', -)\\ w < C{V{x) + V(x')} . 

When the first inequality in ([6]) holds, then it also holds by replacing W with W := 1 
which trivially satisfies the second inequality in ([6]). In this setting Proposition 12.21 is 
Theorem 13.4.4]. Nevertheless, the interest of satisfying the second inequality with an 



unbounded function W is that this allows for control of \\P n (x,-) — P n {x\-)\\w with a 
stronger norm than total variation. Proposition 12.21 also provides a rate of convergence 
for ||P n (x, •) — P n {x\ -)\\w to zero; this rate is stronger than that which could be deduced 



from the control of ||P n (x, •) — ir\\w under similar assumptions [15l . Chapter 14]. 

The following two results show that (J6j) holds when we have a subgeometric-type drift 
inequality, or when we are able to control modulated moments of the return time to a 
small set C. A proof of Proposition 12.31 can be found in [l|, Lemma 5.9]. 

Proposition 2.3. Assume that there exist a set C, a constant b < oo, a measurable 
function V : X — > [1, oo) and a continuously differentiable increasing concave function 
4> : [1, oo) — > (0,oo), such that 

PV < V - <j) o V + 61 c , sup V < oo , inf <j> > . 

C [l,°o) 

Then holds with W oc <f> o V. 

Proposition 2.4. Assume that there exist a set C, a non decreasing rate function r : N — > 
(0, oo) such that r(0) > 1 and sup^^E^ [X]fe=i r (^)] < 00 • Then holds with 



V(x) = E a 



<JC 



k=0 



W(x) = E x [r{a c )\ ■ 

2.3. Examples. A phi- irreducible aperiodic chain satisfying the conditions of Proposi- 
tion [2]3] for a small set C, is ergodic at a subgeometric rate [7|. In that case conditions ([3]) 
and © hold with r(k) = k and W ~ 4> o V, for which r" 1 {C(i~ l V/W) ~ C/?- 1 ^/^ o V. 
This yields the following examples of subsampling rate n and the scale function W; here- 
after, d > 0. 

Logarithmically ergodic chains. Assume that (j)(t) ~ c [1 + lnt] Q for some a > and 
c> 0. Then © holds with n(x) > d n^yLjm and W:=[l + lnV] a . 

Polynomially ergodic chains. Assume that <p(t) ~ ct l ~~ a for some a £ (0, 1) and c > 0. 
Then © holds with n(x) > c'V a (x) and := V l ~ a . 

Subgeometrically ergodic chains. Assume that cj)(t) ~ ct[lni] _a for some a > and 
c> 0. Then © holds with n(x) > c'[lnV(x)] a and := V[\nV]~ a . 

The results above are coherent with the geometric case: on one hand, when a transition 
kernel satisfies the drift inequality PV < /3y + Mc, then it also satisfies ([T|) with the same 
drift function V, and n(x) = c for any constant c G N*; on the other hand, when a — > 0, the 
polynomial and subgeometric drift conditions 'tend' to the geometric drift condition (in 
the sense that 4>(t) — > t). From the above discussion, when a — > 0, [c'y Q , F 1 ""] — > [d, V] 
and [d (lnV) a , V(lnV)~ a ] — > [c', V], thus showing coherence in the results. 
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3. State-dependent drift criteria for regularity 



We now discuss how a state-dependent Foster-Lyapunov drift condition is related to the 
existence of a moment of the return time to measurable sets. Such controls are related to 
the regularity of the chain ([l5, chapter 14], 21]) which, under general conditions, is known 



to imply limit theorems such as strong laws of large numbers, mean ergodic theorems, 
functional central limit theorems and laws of iterated logarithm (see [15I . chapters 14 to 
17]). We provide conditions for the control of (subgeometric) moments of the return time 
to a small set, expressed in terms of a family of nested drift conditions (Proposition 13. lj ) 
or in terms of a single drift condition (Theorem! 



3.1. Family of nested drift conditions. Proposition 13.11 extends the conditions pro- 
vided by Tuominen and Tweedie 2l|, expressed in terms of the one-step transition kernel, 



to the case of the state-dependent transition kernels. 

Proposition 3.1. Let f : X —> [l,oo) and n : X —> N be measurable functions and 
{r(k),k > 0} be a non-negative sequence. Assume that there exist measurable functions 
{Vk,k > 0} and {Sk,k > 0}, Vk,Sk '■ X — > [l,oo), and a measurable set C such that for 
any k > 0, x G X, 



Ex [Vfc +n (* )($ n (* ))] < V h (x) -E a 



n(* )-l 

r{k+j)f{<$> 3 

3=0 



+ S k (x)l c (x) 



Then for any i£X, E x 



Elfo 1 r(k) /(* fc ) < V (x) + S (x) l c (x). 



However, this criterion is more of theoretical than practical interest since it is quite 
difficult to check. We now propose a criterion based on a single drift condition. 



3.2. Single drift condition. We consider the case when 



E x [W(<f> nm )] <(3W(x) + bl c (x) , 



(7) 



for some (3 G (0, 1) and measurable positive functions n, W > 1. The case n{x) = c on X 
corresponds to the usual Foster-Lyapunov drift condition (see e.g. [15I . Chapter 15] and 
references therein). The drift condition extends earlier work by Meyn and Tweedie Q, 
Theorem 2.1] who address the cases when the drift condition is of the form 

E x [W($ n($o) )] <W(x)+bl c (x) , 

K x [W($ nm )) < W{x) - n{x) + btc(x) , 

Ec [W(*„(* ))] ^ P n{x) W(x) + blc(x) , P G (0, 1), (8) 



without assuming any relations between n and W. In it is established that for a 
phi-irreducible and aperiodic kernel P, these drift inequalities imply respectively Harris- 
recurrence, positive Harris-recurrence and geometric ergodicity provided the set C is small 
and W is bounded on C. Our drift inequality ([7]) differs from ([8]) in the rate of contraction 
(3 which does not depend on the subsampling rate n{x). 

Theorem 3.2. Assume that there exist measurable functions W : X — > [1, co) and n : X — > 
N*, constants (3 € (0, 1) and b < oo, and a measurable set C such that holds. If there 
exists a strictly increasing function R : (0, oo) — > (0, oo) satisfying one of the following 
conditions 

(i) 1 1— > R{t)/t is non-increasing and Ron < W , 

(ii) R is a convex continuously differ entiable function such that R' is log-concave and 
R' l (W) -Br x {pW) > n, 

then there exists a constant C such that ~R x \R(tc)} < C{W(x) + btc(x)}. 

The conclusion of Theorem 13.21 is unchanged if R is modified on some bounded interval 
[0,t\: hence it is sufficient to define R such that the above conditions on (R, n,W) hold 
for any x such that n{x) - or equivalently VF(;c) - is large enough. We provide at the end 
of this section examples of pairs (n, W) and the associated rate R. 

A sufficient condition for the existence of R satisfying (JI|) is that, outside some level 
set of n, there exists a strictly increasing concave function £ such that £ o n = W. Then 
we can set R = £. Since concave functions are sub-linear, the case addresses the case 
when n ^> W (outside some bounded set). A sufficient condition for the existence of a 
function R satisfying R" 1 (W) - R~ Y {f3W) > n is that (1-/3) W{x) [iT 1 ]'^^)) > n(x); 
when there exists £ such that 77,(2;) = £ o W(x) and t *— > £,(t)/t is non-increasing, we can 
choose R~ l (t) ~ u" l ^{u)du. Hence case (jn|) addresses the case when n/W is decreasing 
(outside some bounded set). 

Existence of an invariant probabilit y d istribution is related to E x [rc], the first moment 
of the return time to a small set C ( la, Theorem 10.0.1]). Theorem I3.2t ji|) shows that 
the control of this moment can be deduced from a condition of the form (|7|) provided 
n(x) < W(x) (choose R(t) = t). 

Given (n, W) and a drift inequality of the form ([7]), Theorem 13.21 provides a moment of 
the return time to C which depends upon the initial value x at most as W{x) (outside C). 
From the drift inequality (JT]), we are able to deduce a family of similar drift conditions 
with n unchanged: for example, Jensen's inequality implies that for any a € (0, 1), 



E x [W a (<S> n{iS>0) )] < (3 a W a (x) + b a l c (x) 
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Application of Theorem l3.2l with this new pair (n, W a ) , will allow the control of a moment 
of tc which depends on x at most as W a {x). 

Corollary 3.3 (of Theorem 13. 2p . Assume in addition that P is phi-irreducible and aperi- 
odic, C is small with sup c W < oo, and R is a subgeometric rate. 

(i) For any accessible set T>, there exists C < oo such that E x [i?(r-p)] < C W(x); 

(ii) If P admits a unique invariant probability measure tt such that ir(W) < oo, there 



exists an accessible small set V such that sup^-pE^ Y^kL^ 



< oo. 



Examples of moments that can be obtained from Theorem 13.21 (possibly combined with 
Corollary I3.3[) are the geometric, subgeometric, polynomial and logarithmic rates. 

Geometric rates. If nix) = 1: setting R (t) = ln(t)/ln(/c) with 1 < k < /3" 1 it is 
easily verified that condition (Jn|) of the theorem is satisfied. We therefore deduce that 
E x [R(t c )] = E x [k tc ] < C{W(x) +bt c }. In particular, E x \fl~ Tc ] < oo, in agreement with 
the well-known equivalence between the geometric drift condition and the exponential 
moment of tc mentioned in Section [TJ 

Subgeometric rates. If n(x) oc [lny(j;)] a for some a > and W oc I/[lny] _a : then 
n(x) ~ ^ o W{x) with £(i) ~ [lnt] Q . For some convenient c, the function R{t) ~ 
exp(ct 1 /( 1+Q )) satisfies the condition [R^ 1 ]'^) ~ £{t)/[{l - 0)t\ and also condition © 
of Theorem 13.21 

Polynomial rates. If n(x) oc V a (x), for some a € (0, 1] and W oc V 1 ~ a : when a < 1/2 
(respectively a > 1/2) condition ([n]) (resp. condition (jlj)) of Theorem 13.21 is satisfied with 
R(t) ~ We thus have E x [r^"" 1 ] < C V l ~ a {x). 

Logarithmic rates. If nix) oc V[ln V(x)]~ a for some a > and W oc [lnl/] a . Then n ^> 
W and condition © is verified with R(t) ~ [lnt] a . Hence E x [(lnr c ) a ] < C [1 + \nV] a (x). 

As an application of Corollary 13.31 and of the discussion in Section 12.31 we can deduce 
moments of the return-time to C from a single drift condition of the form PV < V — <j) o 
V + blc, 4> concave. For example, in the case <j)(t) ~ t l ~ a for some a 6 (0, 1), we have 
Ex[r c 1/a ] < CV{x). This is in total agreement with that which has been established in the 
literature using other approaches 1Q, [si, Q]. This agreement illustrates the fact that the 



sufficient conditions provided in Sections [2] and [3] are quite minimal. 

4. Application to tame chains 

4.1. Tame chains. The class of tame Markov chains was introduced by Connor and 
Kendall Q], who showed that a perfect simulation algorithm exists for such chains. 



Definition 4.1. The chain $ is tame if there exists a scale function W : X — > [1, oo), a 
small set C, and constants (3 € (0, 1), b < oo such that the following two conditions hold: 
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(i) there exist 8 € (0, 1) and a deterministic function n : X — > N satisfying 



n(x) < W 5 {x) 
such that E x [W($ n{x) )} < (3W(x) + bl c (x) ; 



(10) 



(9) 



(ii) the constant (5 in Q satisfies ln/3 < <5 1 ln(l — 5). 

In other words, <3? is tame if for all x € X there exists a deterministic time n(x) such that 
the chain subsampled at this time exhibits a geometric drift condition (|1U|) . Furthermore, 
n(x) should be sufficiently small compared to the scale function W Q. Part (jn]) of 
Definition ^. H is a technical condition required for construction of the simulation algorithm. 

Clearly the class of tame chains includes all geometrically ergodic chains. In addition, 
it is shown in Q] that <I> is tame if it satisfies a polynomial drift condition of the form 



where < a < 1/4. However, this condition is not necessary: in [2| there is an example 
of a random walk satisfying (jlip with a = 1/2, which is explicitly shown to be tame. The 
results of Section 12.21 now enable us to generalise this sufficient condition. 

Proposition 4.2. Suppose that <3? satisfies the assumptions of Proposition [Ql with <j>(t) ~ 
ct l ~ a where a E (0, 1/2). Then $ is tame. 

Proof. Choose 5 & (0, 1) such that 5 > a/(l — a), and then (3 £ (0, 1) such that ln/3 < 
5" 1 ln(l — 5). As noted in Section 12.31 the results of Proposition 12.31 and Theorem 12.11 
show that if <j>(t) ~ ct l ~ a for a < 1/2, then E x [W($ n(a;) )] < (3W(x) + bl c (x) with 
n(x) = c/3 V a (x) , W = V x ~ a , and where cp oc P^ 1 . The choice of S and (5 ensures that 
$ satisfies all parts of Definition 14. 1\ as required. □ 

Note that any chain with subgeometric drift <j)(t) ~ ct[lnt]~ a is tame. However, the 
logarithmically ergodic chains identified in Section 12.31 do not satisfy <j)(t) > t a for any 
value of a € (0, 1), and so are not covered by Proposition 14.21 

4.2. Dominating process for tame chains. In this section we describe a non-trivial 
example of a Markov chain D for which there is no obvious one-step drift, but for which 
it is simple to establish a sub-sampled drift condition. The chain D finds application in 
the perfect simulation algorithm of [2J], as will be explained below. 

Let f3 £ (0, e _1 ), k > and n* be a function from [1, oo) — > N. To begin our construction 
of D, let U be the system workload of a D/M/l queue, sampled just before arrivals, with 
arrivals every ln/3 -1 units of time, and service times being independent and of unit rate 
Exponential distribution. This satisfies 



PV < V - cV {1 ~ a) + bt 



c , 



(11) 



= max 



{U n + E n+1 - In /T\ 0} 
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where {-Enjn^i ''~ ' Exp(l). 

Define Y = nexp(U), for some k > 0. The set [re, k//3] is a small set for Y, and 

Bu 

F[Yi >v\Y = u} = —, forv> max{0u, as}. (12) 
v 

Finally, let D be the two-dimensional process D = (Z,M) on X := {(z,m) : z G 
[1, oo),m G {1, . . . , n*(^)}} with transitions controlled by: 

P[Z fc+1 = Z fc , M fe+1 = M k - 1 | Z fe , Afjt] = 1 , if M fc > 2 ; 
P[Z fc+1 G £ | Z fc = z, M k = 1] = P[Yi G £ | = z] , 

for all measurable EC [l,oo); 
P[M fe+1 = n*(Z k+1 ) | Z fcj Z fc+1 , M k = 1] = 1 . 

Thus the first component of D is simply a slowed down version of Y, and the second 
component is a forward recurrence time chain, counting down the time until the first 
component jumps (determined by the function n*). 

Proposition 4.3. Let (3 G (0,e _1 ), re > and n* be a measurable function n* : [1, oo) — > 
N. Set C := {(z,m) : z G [re, re//3],m G {1, •"■ i n *{ z )}} anc ^ denote by re the return time 
to C for the Markov chain D. Let cap be the unique solution in (0, 1) of the equation 
In (3 = ln(l — a) /a. 

(i) When n*(z) ~ z 1 for some 7 > : for any a G (0, ap) and 

• any n G (7/a, 1] w/ien 7 G [0, 03) 

• any > 7/a u>/ien 7 > a«, 

£/jere exists a constant C such that Mr gm -\ [t^ 11 ] < C z a for any (z, m) G X. 
(ii) When n*(z) ~ [lnz] 7 /or some 7 > : for any a G (0, ag) and n > satisfy- 
ing T] < {(1 + 7)a~ 1 ln((l — a)/ P")} 1 ^ 1 ^, there exists a constant C such that 
E(^ m )[exp(77ar ( ! // ^ 1+7 ^)] < C z a for any (z,m) G X. 
(Hi) When n*(z) ~ 1, for any a G (0,0/3) there exists a constant C such that for any 
(z,m) G X, E (2 , m) [{(l - a)/r a } TC ] < C z a 

Proof. We first of all establish a drift condition of the form ([7]) for the chain D. Let 
V(z, m) = z a , with a G (0, op). Then E[V(Z m , M m ) \Z = z, M = m] = E[Yf \ Y = z\. 
When z £ [re, re//3], 



f°° Bz 

E[V(Z m , M m ) \Z = z, M = m]= y a ^ dy 

Jbz v 



(3z I - a 



Since a < ap, aln/3 < ln(l — a) and the chain D satisfies the drift condition 

P m V(z, m) < (3'V(z, m) , with = -J— < 1, (13) 

1 — a 
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whenever z £ [k, k/0\. If z G [k, k/B] however, then 

f°° Bz ( Bz 
E[V(Z m ,M m ) | Z = z,M = m}= y^dy+[ 1 - 

Jk y \ K 

\ 1 — a 

It follows that D satisfies P m V(z,m) < B'V{z,m) + b'l c . 

We may now apply Theorem 13.21 to establish moments of the return time of D to the 
set C. For example, suppose that n*(z) ~ z 7 , for some 7 G [0,ag) and set i2(z) := z 1 ^ 
for some 77 G (7/ct, 1]. It follows that i? satisfies the conditions of Theorem 13. 2tjii|) . with 

m)) - iT 1 ^'^, m)) - m > F^/) - RT^iP) - n*(z) 

= (1 - /3 /r >) z ar > -z 1 >0. 

(Here we have used the fact that m < n*(z), by definition of X.) Thus, 

E M [^ /ri ]<C{z a + b't c (z,m)}. 

If n*(z) ~ z" 1 with "f > ap then the same argument shows that the function R(z) = z 1 ^ 1 , 
with 77 > 7/a, satisfies Theorem I3.2l ji)). Parts (ii) and (iii) follow similarly by taking 
R(z) ~ exp(r/a2; 1// ( 1+7 )) and -R(z) ~ exp(rya;z) (for some t] > 0) respectively. □ 

In Proposition I4.3f i). 1/?? > 1 iff 7 G [0, ag). When l/i] > 1 and D is phi-irreducible, 
aperiodic and C is small, this shows that D possesses an invariant probability distribution 
and is ergodic. The convergence to it (in total variation norm) occurs at the polynomial 
rate n 1 ^" 1 (see 0). When I/77 < 1, we cannot deduce from the control of this moment 
the existence of ir. 

The chain D is of interest for the following reason. Suppose that $ is a tame chain 
satisfying P n ^W(x) < pW(x)+bt c > where n(x) = n*oW{x) < W s {x) for some 5 G (0, 1). 
Connor and Kendall Q] show that the chains Z and W(<£) can be coupled so that Z 
dominates W(4>) at the times when Z jumps. Thus the chain D 'pseudo-dominates' 
W($), and this coupling can be exploited to produce a perfect simulation algorithm for 
Proposition 14.31 allows us to calculate ergodic properties of D, and hence bound the 
expected run-time of the algorithm: this issue is not addressed in [2j. 

5. SUBGEOMETRIC ERGODICITY OF STRONG MARKOV PROCESSES 

In this section we provide sufficient conditions for ergodicity of a strong Markov process. 
In 0,HH, the conditions are (mainly) expressed in terms of a drift inequality on the 
generator of the process. Our key assumption AQ] is in terms of the time the process 
rescaled in time and space enters a ball of radius p, p G (0, 1). Proposition 15.11 finds 
application in, for example, queuing theory as discussed below. 
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Let {<3?t,i S K+} be a strong Markov process taking values in X C W 1 . It is assumed 
that ($7, A, J~t, $f , P x ) is a Borel-right process on the space X endowed with its Borel cr-field 
£>(X). We assume that the sub-level sets {x € X, \x\ < £} are compact subsets of X (| • | is 
a norm on R d ). 

Al lim^oo \x\~(p + VE x [|*l*o|*| 1+t J l P+1 ] = for some *o > 0, p > and < r < p. 
A2 For any > 0, there exists C such that for any x G X, sup t<tir f P t (x, dy)\y\ p+1 < 
C\x\p +1 . 

A3 Every compact subset of X is small for the process and the skeleton P is phi- 
irreducible. 



A4 There exist q > and C such that for any x, ~E X 



eB !|1+tj " 1 i^H <c\x\p+k 



Recall that a set C is said to be small (for the process) if there exist t > and a measure 
v on B(X) such that P*(s, •) > I c (x)u(-). 

AQ]is a condition on the process {<]?(, t > 0} rescaled in time and space. Such a transfor- 
mation is largely used in the queueing literature for the study of the stability of networks. 



This approach is refered to as the fluid model (see e.g. 2CJ, Ll3f] for a rigorous definition; 
see also 0, [j], and references therein for applications to queueing) . In these applica- 
tions, AQ] is proved by showing that the fluid model is stable (see e.g. Proposition 
5.1]). Condition A[2]is a control of the L p -moment of the system. A[3]is related to the phi- 
irreducibility of the Markov process, a property which is necessary when ergodicity holds. 



AH] is required to prove the existence of a steady-state value for the moments K x 
s > 0, when t — > oo. Examples of Markov processes satisfying A[T]|3]are given in 



*tlf] for 



4, 



Proposition 5.1. Assume that v4[7H3 hold. Then the Markov process possesses a unique 
invariant probability ir and for any a; € X, limt^ 00 (i-|-l)( p- ' r ) / '( 1+ ' r ' 1 ||P*(x, •) — 7t(-)||tv = 0- 
If in addition holds, then j \y\ q ir(dy) < oo and for any x € X and any < k < 1, 

lim (t + i)k(p-t)/{i+t) gup [g{Xt)] _ n{g) | = o . 

{9 : |9(^)i<i+i^|( i - K 'n 

Proposition 15.11 provides a polynomial rate of convergence and convergence of power 
moments. More general rates of convergence can be obtained by replacing in AQ312] the 
power functions |x| 1+T , by more general functions W; more general moments can 

be obtained by replacing the power function \x\ q in AH] by a general function /. These 
extensions are easily obtained from the proof of Proposition 15.11 details are omitted. 

Proposition 15 . 1 1 extends 0, Theorem 3.1] which addresses the positive Harris-recurrence 
of the process. It also extends [j], Theorem 6.3] by providing (i) a continuum range of 
rates of convergence (and thus a continuum range of rate functions) and (ii) an explicit 
norm of convergence. 
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Proof, of Proposition I5.il The reader unfamiliar with basic results on Markov processes 
may refer to lfj|. For a measurable set C and a delay 5 > 0, let tc(S) := inf{i > i, $( 6 C} 
denote the 5-delayed hitting-time on C; by convention, we write re for rc(0). Let j3 G (0, 1) 
and set W(x) := 1 + jx| p+1 . By AQ3 there exists t > such that t§l G N and for any 



x £ C := {x, |x| 1+T < £}, E x 



<I> 



L*oN 1+r J 



ip+i 



< 0.5/3 |x| p+1 . We can assume without loss 



of generality that I is large enough so that K x [W ($ \t \ x \i+-r \ )] < (3W(x) for x C. Set 
n(x) := max(fto) L^oM 1+r J)- By AO there exists b < oo s.t. 



Ex [W(*„(*))] < /?W(x)+61 c (x) 



(14) 



By Theorem there exists C < oo such that E x [{t* iC } (p+1)/(t+1) ] < CW(x) where 
r*c := inf{?i > 1, G C} is the return time to C of the skeleton P. Hence, there exists 
a delay < 5 < 1 such that E x [rc(<5)] < CW(x); sup c W < oo and C is small for the 
process (by A[3j) , so {<I>t, i > Ojis positive Harris-recurrent and possesses a unique invariant 
probability measure it. By 17, Proposition 6.1] and jl5, Section 5.4.3], the skeleton P is 
aperiodic and any compact set is small for the skeleton P. A[3]and the above properties on 
the skeleton P imply lim neN (n + 1)(p~^)/( 1 + t ) \\P n (x, ■) - tt(-)||tv = for any x G X [21 



Theorem 2.1], which in turn implies that lim tgR+ (t + 1)(p- t )/( 1+t ) ||P*(x, •) — vr(-)|| TV = 0. 
Set f{x) := 1 + \x\ q . AH and CO} imply that 



< C W(x) 



where tq is the return time to C of the discrete-time chain {<!>„, re > 0} with transition 









>c-l 








-?c-l 




E /(**) 






E /(**) 




<CE X 






fc=0 




fc=0 


fc=0 






. k=0 



kernel P n ( x )( 



Hence vr(/) < oo by [15I . Theorem 14.3.7]. As in |7j, we obtain (/, r 



a, 



modulated moments of r^c by using Young's inequality (see e.g. [ll[). This yields ergodic 
properties for the skeleton P and the desired limits for {P , t > 0}. □ 



6. Appendix: Proofs 



6.1. Proof of Proposition 12.21 In the proof, C is constant and its value may change 
upon each appearance. We use the following properties 

Rl If T> is petite for a phi-irreducible and aperiodic transition kernel, then it is also 
small (|15l. Theorem 5.5.7]). 



R2 If a transition kernel is phi- irreducible and aperiodic, then any skeleton is phi- 
irreducible and aperiodic (Q, Proposition 5.4.5]). 

R3 If T> is z^-small for a phi-irreducible and aperiodic transition kernel, then we can 
assume without loss of generality that v is a maximal irreducibility measure (Q, 
Proposition 5.5.5]). 
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R4 If there exist measurable functions f , V : X — ► [l,oo), a measurable set C and 

n 

a constant b such that PV < V — f + 61c, then for any stopping-time r ([la. 
Proposition 11.3.2]) 



Ex M < 



T-l 



£/(**) 



fc=0 



< V(s) + 6E X 



T-l 



fe=0 



If in addition, there exist m > 1 and c > such that clc(x) < P m (x,D), then 



fa] < E x 



T-p — 1 

E /(**) 

fc=o 



< V[x) + - E a 



'ru — 1 

E l2?(*fc+m) 
. k=0 



< V(x) + 



b(m + 1) 



The conditions ([6]), sup c V < oo and B[4] imply that sup xgC E x [rc] < oo. Hence, C is an 
accessible set. By RQ]and R[3] there exist m > 1 and a maximal irreducibility measure i/ m 
such that P m (x, •) > t c {x)v m {-). 

Step 1: From P to the strongly aperiodic transition kernel P m . By Q, there 
exists a constant C such that W^ m) := YT=o pkw <C W. Write n = km + 1 for k G N 
and Z G {0, • • • , m — 1} so that 



n \\P n (x,-) - P n (x', 



< n \\P km (x,-)-P Km (x',- 



km (I 



< C (fc + 1) ||P fcm (x,-) -P fc "V,-)|| 



ir 



By H2J P m is phi-irreducible and strongly aperiodic, and satisfies drift inequalities of the 
form ©. Indeed, 



m— 1 



m— 1 



m— 1 



P m V{x) < V(x) - E P h W(x) + 6 E C ) ^ " + b E C ) ; 

fc=0 fc=0 fc=0 

this implies that there exist a small set T> (for P m ) and a constant 6 < oo such that 
P m V{x) < V{x) - 0.5 W{x) + Mp(i) (see e.g.Q, proof of Lemma 14.2.8]). By BHJ for 
any accessible set A (for P m ), there exists C such that 

Ex[ri m) ] < CV{x) 

-M ._ 

-4 •" 

bY2k=o P k { x ,G)- This implies for any accessible (for P m ) measurable set A 



(15) 



where t^ 1 ' := inf{n > 1, $ nm G „4}. Furthermore, we also have P m W(x) < W(x) + 



+ 1) E x 



k<r 



(m) 



< C7(x) ; 



(16) 



(the proof is postponed below). Following the same approach as in [2JJ] or fl5J, Chapter 
14], we use the splitting technique and associate to the chain with transition kernel P m a 
split chain that possesses an atom. 

Step 2: From P rn to an atomic transition kernel P. A detailed construction of the 
split chain and the connection between the original chain and the split chain can be found 
in [15I . Chapter 5]. We use the same notation as in 15]. The split chain {(<!>„, <i n ), n > 0} 
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is a chain taking values in X x {0, 1}: its transition kernel is denoted by P. Based on the 
connection between P m and P, Proposition 12.21 holds provided there exists C < oo with 

m(n + l) sup \P n f(x)-P n f(y)\ < C {V(x) + V(y)} . (17) 

{/,sup x |/|[W]-l<l} 

We prove (|17|). P is phi-irreducible and aperiodic and possesses an accessible atom a := 
C x {1}. Let T a be the return time to the atom a. From (|15p . we have E x * [r a ] < CV(x) 
(see e.g. 0, Proposition 3.7] or Lemma 2.9]); and by (fl~6|) 

(k + 1) E x * [W(<f> k )l k < T J < CV(x) . (18) 

(The proof of (fl~8j) is postponed below.) 

Set a x (ri) := P x *(tq, = n), u(n) := P a (($ n ,d n ) € a) and t f (n) := E a [/($ n )l„< r J, where 
a * b(n) := J2k=o a (k)b(n — k). Then, by the first-entrance last-exit decomposition 
Chapter 14], for any function / such that |/| < W, 
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(n + 1) \P n f(x) - P n f(y)\ < (n + 1) E x * [|/|($„)1„< T J + (n + 1) Ej,* [|/|(*„)l n < Ta ] 

+ (ro + 1) |a x * it — a y * u| * %|(^) • 

By (I18p . the first two terms in the right-hand side are upper-bounded by C{V(x) + V(y)}. 
Applying again JTS]), sup fc > x k < sup^ k E a [W($ k )l k < Ta ] < Csup c V < oo, so 

(n +1) |a a * u — a a * u\ * tf(n) < sup k t\y{k) sup(n + 1) \a x * u — a y * u\ (n) 

\k>l J n>l 

< C sup(n +1) \a x * u — a y * u\ (n) . 

n>l 



Since sup c V < oo, E a [r Q ] < oo; standard results from renewal theory imply (see e.g. 121]) 
sup n>1 (n + 1) \a x * u — a y * u\ (n) < C{E X * [r Q ] + E y * [tq,]}. The right-hand side is upper 
bounded by C{V(x) + V(y)}. This concludes the proof of (fT7j) and thus the overall proof. 
Proof of inequality (JE). Using P"W(x) - < fe^"!^ 1 



(n+ 1) E x nm)l 



J™) 



W(x) 



n 

[(k + 1)^(^)1^ 



k=l 



) -W($(fe-l)m)l M < r W 



n 

<-Y. 

k=i 


E. [W($ km )l 


k<T<r>] + 


n 

^E x [k{W($ km ) - W($ (l 

k=l 
"r (m) 


i-l)m)}l fc _ 1 < T (m)_ 


<E X 


k=l 


+ bE x 


T A m—l 
k =0 j=Q 


(19) 



Since ^4 is an accessible set for P m and P is small for P m , there exist c, r > such that 
Clx>(x) < P mr (x,^4). Hence, by E[H the first term in (p~9|) is upper bounded by CV(a;). 
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P is aperiodic and C is small: there exist I > 1 and a non trivial measure v such that 
v{A) > and P lm ^(x,A) > l c (x)v(A) for any j € {0, • • • ,m - 1} (see e.g. , proof of 
Lemma 14.2.8]). Hence, P lm (x,A) > P j (x,C)u(A) for any j G {0, • • • , m - 1}, and this 
yields mP lm (x,A) > P j {x,C)v{A). Therefore, there exists C < oo such that 



m— 1 



]T P j (x,C) < C P /m (x,„4) = C E x [l A {$ 

j=0 



Im ) 



This yields 



6E, 



r .4 



m— 1 



<CE T 



<CE X 



r (m) 



£> + l) E$ fem [l^(*, m )] 



fc=0 



£ (k-l + 1) 1 A ($ 



km ) 



k=l 



<CE, 



^2 {k-l + l) t A {$k, 



km ) 



CE X 
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km) 



<C {E a 



r ^4 



+ 1} < CV(x) . 



Proof of inequality (|18p . In the sequel, we write re as a shorthand notation for 
T Cx{o,i}' anc ^ ^ ^ f° r {^i ^ C, • ••,<&„ ^ C}. Let {r 9 , g > 1} be the successive return 
times to C x {0, 1}. We write 



(k + 1) E x * [W($ k )l k < T J = (k + 1) [W/($ fe )l fc < Wa=rc ] 

?>1 L C 

The connection between P m and P yields: 

(fc + 1) E x + [W($ k )l k < TatTa=TC ] <(k + l) E x [W($ km )t k ^ m) ] < CV(x) . 
For the second term, let q > 1 and consider a general term of the series: 



• (20) 



(k + 1) E x 



w^ k )t Tl<k ^ T t K ic 



k-l 



1=0 



< J> + 1-1) E x * [l T , = ,l l<Ta E* Iid| [W^k-Ol^^fc 
fc-i 



(21) 
(22) 



17 



By definition of the split chain, 

(k + 1- l)E x * [l T « = ,l;< TQ t^ dl [Wpk-fiH^^tc 

= E x + [t TH t l<Ta Ez lA [E M [(k + 1- I) W^k-i-!)!^^]] 

< E x * [l T , ={ l I<Ta E ftlidl [e $1 [(k + 1- I) H^($ rn(fc _ / _ 1) )l T ( m) ^ fc _ i _ 1 

< C E x > [l TH l l<Ta E^ A [V(^)]\ < swpRV F x * (l TH t l<Tc ^ . 

Hence, (pZTj) is upper bounded by C E x * (tJ < r a ) and thus by C (1 — e) q ~ l . Furthermore, 
(f22j) is upper bounded by 

fe-i 



C SUpRW J2 l ( T C = l , l < T a) < CEa 



T C K$<Ta 



1=0 

q _ v-i?-lr r+1 



The decomposition = X^=i{ r c ~~ r c}+ T c> an d the inequalities P^*^ < r a ) < (1— e) 



and sup Cx { 0)1 } E X)d [t c ] < oo, imply that E a 



<C(l-e)« E x *[r c ]. 



The second term in the rhs of (|20|) is a convergent series. This concludes the proof. 

6.2. Proof of Proposition 12.41 Under the stated assumptions on r, infx V > 0, and 
infx W > 0. For any measurable set C and any function / : X — > [l,oo) the function 
F(x) :=E X [Efei /( $ fc)] satisfies 



PF(s) = E a 



£/(**) 



.fc=i 



- f(x) + t c (x) E x 



£/(**) 



,fc=l 



<F(x)-/(x)+6l c (x) 



where 6 := sup^E^ [Efcli 



6.3. Proof of Proposition I3TT1 Upon noting that r, / are non-negative and that r fc+1 



T k + TO 6 T 



-a.s. 



E, 



T C -1 

E r (*o 

.fc=0 



< E 7 



E r (*o /(** 



fc=0 



< E 7 



E E r(j + r k ) f($> j+Tk 
k=0 j=0 



By definition of the random time r, r 



n($ T fc) and this implies 



E, 



TC-l 

E r (*o 

fc=0 



< E T 



E, 



V c _in($ T fc)-l 

E E ^ + r fe )/($, +Tfc 
/o=o j=o 



fc-l 

E E 

fc=0 Z>fc 



n(*o)-l 

E Ki+o 

3=0 



By the drift assumption, P^-a.s. , 



E,, 



n($ )-l 

E rti + l)f(*i) 

j=0 



< 



V^-E^ [^+n(*„) (**(*„))] +5,(*,)lc(*l) 
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SO that 



E, 



< E, 



k=0 



Tc—i r r c _i 

E r ^ 

fc-1 

E {V T k (*rk) - F T fc+l ($ r M-i ) + 5 rfc ($ r fc ) lc (*r*)} 
fc=0 



E {^ T *(*T*) " ^+n(<I. Tfe )( $ rfe+n($ Tfc )) + S T h ($ r fc)l C ($ r *)} 

<Fo(^) + 5o(x)lc(x) . 



6.4. Proof of Theorem 13.21 Define r := n($o) and the iterates r 1 := r, r fc+1 := 
r o # T + r k for > 1, where 9 denotes the shift operator. (By convention, r° = 0.) 
Finally, set fc ■= inf{& > 1, <£ r fc € C}. 

Proof of Theorem \3.2X i\). By definition of the random time r, P x -a.s. , re < r rc = 
X^fc^o 1 n (^T fe )- Since i i— > R(t)/t is non- increasing, we have i?(a + b) < R(a) + R(b) for 
any a, 6 > 0. This property, combined with the fact that R is increasing yields P x -a.s. 



'i-i 



l-i 



rc-l 



2>1 



A;=0 



fc=o 



where we used R(n(x)) < in the last inequality. The proof is concluded upon noting 

that from the drift assumption and the Comparison Theorem [15I Proposition 11.3.2], 



;i - p) e 2 



TC — 1 

E 

. k=0 



< W{x) + bl c {x) 



Proof of Theorem 1 3. Mvi\) . Under the stated assumption, the inverse H := R 1 exists. 
Set r(t) := R'(t) = 1/(H' o R(t)). Define a sequence of measurable functions {H k , k 6 N}, 



H k : [1, oo) -» (0, oo) by £T fc (t) := /„ H W r(z + k) dz = R(H(t) + k) - R(k). Then F fc 
increasing and concave. Indeed H' k {t) = and this is positive since i? is increasing. 

Since (and thus H) is increasing and 1 1 1— > r(t+k)/r(t) is non-increasing (since i?' is log- 
concave), i7£ is non- increasing. The Jensen's inequality and the drift assumption imply 
for any k € N, x £ X, 

P(H k+n{x) O W($n(a:))) ^ (^(*n(*))) < + W c(*)) 

< #fc+n(x)(Wx)) + F fc+n(:E) (6)lc(x) 

n(:r) — 1 

< H k (W(x)) - E r ( fc + i) + #fc+n(*)(&)lc(z) + K k {x) , 

i=o 

where we defined K k (x) := ^(^(Wx)) - fl^W^)) + EjS" 1 + i)- We now 
prove that lZ k {x) < which will conclude the proof by applying Proposition 13.11 with 



is 
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V k = H k o W. We have 

,-H{pW(x))+n{x) ,-H{W{x)) 

T^k( x ) = ^2 r(j + k)+ I r[z + k)dz — / r(z + k)dz 

•_ n J n(x) JO 



3=0 



n(x)-l n ( a ) 

— r d + ^) _ / r ( z + j 

,-=o ^ 

since by assumption, H(W) > H{(3W) + n. Now, R is convex and so r is non-decreasing. 
Therefore, Y^j=o 1 r C? + ^) — fo^ r ( z + &)<fe an d this concludes the proof. 

6.5. Proof of Corollary 13.31 (a) This follows from jlH Lemma 3.1 ] (see also [21, Propo- 
sition 3.1]). (b) By (15I . Theorem 14.2.11], there exist measurable sets {A„,n > 0} whose 

< 00 for any accessible set T>. For 



union is full and such that sup^g^ E x XlI=o W(^fe) 
an accessible small set P, there exists n such that X> := T> n A n is small and accessible. 

< 00. 



r t — 1 

The proof follows by combining the results of (a) with sup^^E^ X}fc=o W(^fc) 
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